1. A computer-implemented method comprising: receiving, by a first stage hotword detector of a multi-stage hotword detection system that includes at least (i) the first stage hotword detector that is implemented in hardware and that is associated with a memory, and (ii) a second stage hotword detector that is implemented in software and that is configured to output a signal indicating whether a given utterance likely includes a hotword, audio data that corresponds to an initial portion of an utterance; providing, by the first stage hotword detector that is implemented in hardware and to the second stage hotword detector that is implemented in software, and based on the first stage hotword detector determining that the initial portion of the utterance likely includes the hotword, data indicating an instruction to access the audio data that corresponds to the initial portion of the utterance and that is stored in the memory associated with the first stage hotword detector; receiving, by the first stage hotword detector that is implemented in hardware and from the second stage hotword detector that is implemented in software, audio data that corresponds to a subsequent portion of the utterance; receiving, from the second stage hotword detector that is implemented in software and by the first stage hotword detector that is implemented in hardware and based on the second stage hotword detector determining that the initial portion of the utterance likely includes the hotword, data indicating an instruction to cease providing, to the memory associated with the first stage hotword detector, audio data that corresponds to one or more subsequent portions of the utterance; and in response to receiving, from the second stage hotword detector that is implemented in software and by the first stage hotword detector that is implemented in hardware and based on the second stage hotword detector determining that the initial portion of the utterance likely includes the hotword, the data indicating the instruction to cease providing, to the memory associated with the first stage hotword detector, the audio data that corresponds to the one or more subsequent portions of the utterance, providing, for output by the multi-stage hotword detection system, a signal indicating that the utterance likely includes the hotword.