1. A computer implemented method comprising: receiving, at a processing system, a first signal representing an output of a speaker device; receiving, at the processing system, a second signal comprising (i) the output of the speaker device and (ii) an audio signal corresponding to an utterance of a speaker; aligning, by the processing system, one or more segments of the first signal with one or more segments of the second signal; classifying acoustic features of the one or more segments of the first signal to obtain a first set of vectors associated with speech units; classifying acoustic features of the one or more segments of the second signal to obtain a second set of vectors associated with speech units; modifying the second set of vectors using the first set of vectors to obtain a modified second set of vectors, wherein the modified second set of vectors represents a suppression of the output of the speaker device in the second signal; and providing the modified second set of vectors to generate a transcription of the utterance of the speaker.