1. The method of training a barge-in-model for speech processing, the method comprising: receiving an input having at least one speech segment and at least one non-speech segment; establishing a restriction of recognizing only speech states during speech segments of the input and non-speech states during non-speech segments of the input; generating a hypothesis lattice by allowing any sequence of speech Hidden Markov Models (HMMs) and non-speech HMMs; generating a reference lattice by only allowing speech HMMs for at least one speech segment and non-speech HMMs for at least one non-speech segment, wherein different iterations of training generates at least one different reference lattice and at least one reference transcription; and employing the generated reference lattice as the barge-in-model for speech processing.