Chen2019BERTFJ use a Transformer [1]} model for joint SF and IC by fine-tuning a pre-trained BERT [2]} model (Figure REF  Right). The input is passed through several layers of transformer encoders and the hidden state outputs are used to compute slot and intent labels. The hidden state \(h^{\texttt {CLS}}\)  is used for IC[CLS] is a special token in BERT input format that often used as the sentence representation. while the rest of the hidden states at each time step \(h_i\)  serve SF.
