1. A method for reducing latency in speech recognition, the method comprising: receiving audio input data representing an utterance; performing automatic speech recognition (ASR) processing on the audio input data to generate ASR output; determining a first ending to the utterance in the audio input data at a first time corresponding to non-speech detected in the audio input data; determining a first portion of the ASR output, the first portion corresponding to the audio input data up to the first ending; providing the first portion of the ASR output to a natural language understanding (NLU) module to obtain a first NLU result; storing the first NLU result; determining a second ending to the user's speech in the audio input data at a second time after the first time; determining a second portion of the ASR output, the second portion corresponding to the audio input data up to the second ending; comparing the first portion to the second portion; and: (1) if the first portion is the same as the second portion, initiating a first action to be executed on a first device, the first action based on the first NLU result, and (2) if the first portion is not the same as the second portion: discarding the first NLU result, providing the second ASR output to the NLU module to obtain a second NLU result, and initiating a second action to be executed on the first device, the second action based on the second NLU result.