It would be interesting to try different architectures. Long-short term
memory models are often used for natural language processing applications,
as in neural machine translation [1]}, [2]}. However, the
structure of the inputs is really a 2D structure, like an image. So convolutional
neural networks may detect correspondences between co-occurring features of
words that other network architectures would not. An architecture for detecting
whether a pair of words are used metaphorically, called “supervised similarity
networks,” was introduced in [3]}. While pair detection like this
is not directly applicable for our task, the results are promising, and
this approach should be investigated more.
