We evaluate our performance on the main task of question relevancy ranking using the official SemEval-2017 Task 3 evaluation scripts [1]}. The scripts provide a variety of metrics; however, in accordance with the shared task, we report Mean Average Precision (MAP) (the official metric for the SemEval 2016 and 2017 shared tasks); Mean Reciprocal Rank (MRR), which has being thoroughly used for IR and QA; Average Recall; and, finally, the accuracy of predicting relevant documents.
