[1]} employ Long Short-Term Memory (LSTM) networks [2]} to encode the target position information in word embeddings. [3]} transfer latent opinion knowledge into a Bidirectional LSTM (BiLSTM) network that leverages word and position embeddings [4]}. Recently, [5]} have proposed ONG, a method that combines BERT (Bidirectional Encoder Representations from Transformers) [6]}, position embeddings, Ordered Neurons LSTM (ON-LSTM) [7]},An LSTM variant with an inductive bias toward learning latent tree structures in sequences. and a graph convolutional network (GCN) [8]} to introduce syntactic information into word representations. While this model achieves state-of-the-art results, previous studies have shown that the ON-LSTM does not actually perform much better than LSTMs in recovering latent tree structures [9]}. Besides, ON-LSTMs perform worse than LSTMs in capturing short-term dependencies [7]}. Since opinion words are usually close to targets in text, ON-LSTM risks missing the relationship between the aspect and any information (e.g. position) relating to the opinion words.
