To test our hypotheses, we compare three different classifiers: a Support Vector Machine (SVM) with Bag-of-Embeddings feature representations, a Convolutional Neural Network (Cnn) [1]}, [2]}, and a Bidirectional Long Short-Term Memory Network (BiLstm) [3]}. Each of these classifiers theoretically has an increasing reliance on word order. Note that we do not use the bilingual sentiment model [4]}, as it jointly learns both projection and classifier and cannot be used as input to the Cnn and BiLstm. Although the SVM does not take into account word order at all, it is a strong baseline for sentiment analysis [5]}. The Cnn considers only local word order, while the BiLstm relies on both local and long-distance dependencies.
