Recently, <cite>Zhong et al. (2018)</cite> proposed a model based on training a binary classifier for each slot-value, Global-Locally Self Attentive encoder (GLAD, by employing recurrent and self attention for each utterance and previous system actions, and measuring similaity of these computed representation to each slot-value, which achieve state of the art results on WoZ and DSTC2 (Williams et al., 2013) datasets. Although the proposed neural based models achieves state of the art results on several benchmark, they are still inefficient for deployment in production system, due to their latency which stems from using recurrent networks. In this paper, we propose a new encoder, by improving GLAD architecture<cite> (Zhong et al., 2018)</cite> .