1. A method for dynamically adjusting speech recognition processing to reduce latency, the method comprising: receiving a first portion of an audio input corresponding to an utterance; identifying a time stamp associated with the first portion; performing speech recognition processing on the first portion using a first graph pruning factor; identifying a current time of processing of the first portion; determining a current latency of the utterance by comparing the time stamp to the current time; determining a property of a second portion of the audio input prior to performing speech recognition processing on the second portion, the property comprising an estimated difficulty of speech recognition processing, the estimated difficulty based on a percentage of the second portion of the audio input that has a signal to noise ratio below a threshold; determining an estimated latency based at least in part on the property of the second portion and the current latency; comparing the estimated latency to a target latency; determining a second graph pruning factor based at least in part on the comparing; performing additional speech recognition processing on the second portion using the second graph pruning factor; and outputting speech processing results.