CogALex V Shared Task: GHHH Detecting Semantic Relations via Word Embeddings
This paper describes our system submission to the CogALex 2016 Shared Task on Corpus Based Identification of Semantic Relations. Our system won first place for Task 1 and second place for Task 2. The evaluation results of our system on the test set is 88.1% (79.0% for TRUE only) f measure for Task 1 on detecting semantic similarity, and 76.0% (42.3% when excluding RANDOM) for Task 2 on identifying finer grained semantic relations. In our experiments, we try word analogy, linear regression, and multi task Convolutional Neural Networks (CNNs) with word embeddings from publicly available word vectors. We found that linear regression performs better in the binary classification (Task 1) while CNNs have better performance in the multi class semantic classification (Task 2) We assume that word analogy is more suited for deterministic answers rather than handling the ambiguity of one to many and many to many relationships. We also show that classifier performance could benefit from balancing the distribution of labels in the training data.
CogALex@COLING