1. A system configured for segmenting and/or classifying an audio signal from transformed audio information, the system comprising: one or more processors configured to execute computer program modules, the computer program modules comprising: an audio information module configured to obtain transformed audio information representing a sound, wherein the transformed audio information specifies magnitude of a coefficient related to energy amplitude as a function of frequency for the audio signal and time; an audio features module configured to obtain features associated with the audio signal from the transformed audio information, individual ones of the features being associated with a feature score relative to a predetermined speaker model; and an aggregate score module configured to obtain an aggregate score based on the feature scores according to a weighting scheme, the weighting scheme being associated with a noise estimation, the aggregate score being used in conjunction with the predetermined speaker model to identify a source of the sound in the audio signal.