1. A non-transitory computer-readable storage medium storing program instructions, the program instructions being computer-executable to implement: for a first source, generating a model for each word of a plurality of words, each model includes including: a plurality of dictionaries, each of the plurality of dictionaries including one or more spectral components; and probabilities of transition between the plurality of dictionaries; and constraining the models according to high level information that defines valid transitions, the constrained models being usable to perform source separation on a sound mixture that includes multiple sources.