For the encoder layer, various RNN architectures have been applied to SF and IC [1]}, [2]}, [3]}. Mesnil2013InvestigationOR compare the Elman [4]} and Jordan [5]} RNNs. They observe that the performance of the Jordan RNN is marginally better than Elman. They also experiment a bi-directional version of Jordan RNN and obtained the best score of 93.89 F1 for SF, performing better than CRF for about +1 absolute F1 improvement. xu2013convolutional use Convolutional Neural Network (CNN) [6]} to extract 5-gram features and apply max-pooling to obtain the word representation before passing it to the output layer. Compared with RNN [7]}, [1]}, CNN gives lower performance for SF on ATIS. Other studies [9]}, [10]} adapt Long Short-Term Memory Network (LSTM) [11]} to SF. The LSTM model gives better SF performance compared to CRF, CNN, and RNN. Ravuri2015RecurrentNN compare the performance of vanilla RNN and LSTM for IC. They find that the vanilla RNN works best for shorter utterances, while LSTM is better for longer utterances.
<TABLE>