In this work we describe the system built for the three English subtasks of the SemEval 2016 Task 3 by the Department of Computer Science of the University of Houston (UH) and the Pattern Recognition and Human Language Technology (PRHLT) research center - Universitat Polit ecnica de Val encia: UH-PRHLT. Our system represents instances by using both lexical and semantic-based similarity measures between text pairs. Our semantic features include the use of distributed representations of words, knowledge graphs generated with the BabelNet multilingual semantic network, and the FrameNet lexical database. Experimental results outperform the random and Google search engine baselines in the three English subtasks. Our approach obtained the highest results of subtask B compared to the other task participants.
The previous edition of the SemEval CQA task included two English subtasks @cite_7 . The first one was focused on classifying answers as , , or with respect to one question. The second subtask answered a question with , , or based on the list of all answers. In addition, the first subtask was also available in Arabic. Several teams experimented with complex solutions that included meta-learning, external resources, and linguistic features such as syntactic relations and distributed word representations. Similarly to our work, the highest performing approach employed a combination of lexical and semantic-based similarity measures @cite_13 . Its semantic features included the use of probabilistic topic models, translation obfuscation-based alignments, and pre-computed distributed representations of words both generated with the word2vec https: code.google.com archive p word2vec and GloVe http: nlp.stanford.edu projects glove toolkits. Their lexical features included BOW, word alignments, and noun matching. They employed a regression model for classification. Another interesting approach, , included textual features --- word lengths and punctuation --- in addition to syntactical-based features --- Part-of-Speech (PoS) tags.
@cite_13 @cite_7
2252217313 2467646401 2155484602 2251181482
Community Question Answering (cQA) provides new interesting research directions to the traditional Question Answering (QA) field, e.g., the exploitation of the interaction between users and the structure of related posts. In this context, we organized SemEval2015 Task 3 on Answer Selection in cQA, which included two subtasks: (a) classifying answers as good, bad, or potentially relevant with respect to the question, and (b) answering a YES NO question with yes, no, or unsure, based on the list of all answers. We set subtask A for Arabic and English on two relatively different cQA domains, i.e., the Qatar Living website for English, and a Quran-related website for Arabic. We used crowdsourcing on Amazon Mechanical Turk to label a large English training dataset, which we released to the research community. Thirteen teams participated in the challenge with a total of 61 submissions: 24 primary and 37 contrastive. The best systems achieved an official score (macro-averaged F1) of 57.19 and 63.7 for the English subtasks A and B, and 78.55 for the Arabic subtask A. We describe our system for finding good answers in a community forum, as defined in SemEval-2016, ask 3 on Community Question Answering. Our approach relies on several semantic similarity features based on fine-tuned word embeddings and topics similarities. In the main Subtask C, our primary submission was ranked third, with a MAP of 51.68 and accuracy of 69.94. In Subtask A, our primary submission was also third, with MAP of 77.58 and accuracy of 73.39. Shared Task 1 of SemEval-2014 comprised two subtasks on the same dataset of sentence pairs: recognizing textual entailment and determining textual similarity. We used an existing system based on formal semantics and logical inference to participate in the first subtask, reaching an accuracy of 82 , ranking in the top 5 of more than twenty participating systems. For determining semantic similarity we took a supervised approach using a variety of features, the majority of which was produced by our system for recognizing textual entailment. In this subtask our system achieved a mean squared error of 0.322, the best of all participating systems. Continuous word and phrase vectors have proven useful in a number of NLP tasks. Here we describe our experience using them as a source of features for the SemEval-2015 task 3, consisting of two community question answering subtasks: Answer Selection for categorizing answers as potential, good, and bad with regards to their corresponding questions; and YES NO inference for predicting a yes, no, or unsure response to a YES NO question using all of its good answers. Our system ranked 6th and 1st in the English answer selection and YES NO inference subtasks respectively, and 2nd in the Arabic answer selection subtask.