1. A computer-implemented method comprising: receiving, by a computing device that uses voice-based speaker identification, data identifying a user; prompting the user to confirm whether a previously received utterance was properly identified, properly rejected, falsely identified, or falsely rejected using voice-based speaker identification; receiving, from the user, data indicating that the previously received utterance was properly identified, properly rejected, falsely identified, or falsely rejected; and in response to receiving the data indicating that the previously received utterance was properly identified, properly rejected, falsely identified, or falsely rejected, using audio data corresponding to the previously received utterance to perform voice-based speaker identification on subsequently received utterances.