Conventionally, semantic frame parsing can be achieved by a variety of techniques, including conditional random fields (CRFs) [1]}, [2]}, hidden Markov chains (HMMs) [3]} and other machine learning models, including decision trees and support vector machines (SVMs) [4]}. In recent years, many SLU semantic frame parsing models have been built upon recurrent neural network (RNN) models due to their advantages in performing sequence prediction. Some of these models demonstrate state-of-the-art performance on benchmark SLU datasets, such as ATIS [5]} and SNIPS [6]}. Despite the decent performance of these models, however, it is worth noting that most of the queries in these SLU datasets are relatively short (with an average of 11.4 tokens/query for ATIS and 9.02 tokens/query for SNIPS), and the semantic patterns within each dataset are also very similar. The previous RNN models built on these datasets may not perform well for long sentences containing complex and irregular semantic structures, such as Twitter messages. Due to the large variety of different Twitter writing styles, existing RNN models perform poorly at parsing complex Twitter texts. Technically, there are two main reasons for this poor performance:
