1. A computer-implemented method comprising: receiving, by a second stage hotword detector of a multi-stage hotword detection system that includes at least a first stage hotword detector and the second stage hotword detector, audio data that corresponds to an initial portion of an utterance; determining, by the second stage hotword detector, a likelihood that the initial portion of the utterance includes a hotword; determining, by the second stage hotword detector, that the likelihood that the initial portion of the utterance includes the hotword satisfies a threshold; and in response to determining that the likelihood satisfies the threshold, transmitting, by the second stage hotword detector and to the first stage hotword detector, a request for the first stage hotword detector to cease providing additional audio data that corresponds to one or more subsequent portions of the utterance.