To accomplish this MV classification task, we experimented with deep
feed forward neural networks, encouraged by success on a similar task reported in
[1]}. Each hidden layer had 500 nodes, and we
tested 1-, 2-, 4-, and 6-hidden-layer models. To regularize we use
dropout and early stopping. Stochastic gradient descent with momentum
minimized the cross entropy loss. The inputs were vectors with 3300
elements: eleven words represented by their word2vec embeddings, which
come from the Google News pre-trained word2vec
(download).
Each word embedding vector is concatenated with the next to create the
network input vector. We used the free and open source Python package
gensim for loading the binary-formatted word2vec
[2]}, [3]} word embeddings [4]}.
Neural network construction, training, and testing was done primarily in
TensorFlow
[5]}, [6]}. Performance
analysis was done using Scikit-Learn [7]},
and data table reading, writing, and
subsetting was done with Pandas [8]}.
More details can be found by examining the
README and code for
this project on GitHub.
