We consider two sets of commonly used neural models for the tasks of binary classification and multi-class natural language inference. We use CNN and bi-directional LSTM based models with attention. We follow [1]} and use similar attention mechanisms using a) additive attention [2]}; and b) scaled dot product based attention [3]}. We jointly optimize all the parameters for the model, unlike [1]} where the encoding layer, attention layer and the output prediction layer are all optimized separately. We experiment with several optimizers including Adam [5]}, SGD and Adagrad [6]} but most results below are with Adam.
