1. A method for dynamically adjusting speech recognition processing to reduce latency, the method comprising: receiving an audio signal, the audio signal corresponding to an utterance; performing first speech recognition processing on a first portion of the audio signal, the first speech recognition processing involving a first plurality of hypotheses, each of the first plurality having a respective score within a first hypothesis score range; determining a processing value for each frame of the first portion, wherein the processing value corresponds to an amount of processing performed by one or more processors; determining a total number of frames in the first portion; determining, using the processing value, an estimated amount of time to process the first portion; determining that the estimated amount of time is above a time threshold; determining an adjusted hypothesis score range to increase a speed of speech recognition processing in response to the estimated amount of time being above the time threshold; and performing second speech recognition processing on a second portion of the audio signal using, the second speech recognition processing involving a second plurality of hypotheses, each of the first plurality having a respective score within the adjusted hypothesis score range, the second plurality including fewer hypotheses than the first plurality.