1. A method of speech-based user authentication, comprising: at a device having one or more processors and memory: receiving a plurality of different spoken utterances from a user; for each of the plurality of different spoken utterances: generating a respective phoneme-independent representation from the spoken utterance, the respective phoneme-independent representation including a respective spectral signature for each of a plurality of frames sampled from the spoken utterance; and decomposing the respective phoneme-independent representation to obtain a respective content-independent recognition unit for the user; calculating a content-independent recognition distribution value for the user based on the respective content-independent recognition units generated from the plurality of different spoken utterances; and providing the content-independent recognition distribution value for use in a speaker authentication process.