Experimental setup Previous research relies on manual annotation to evaluate the ranking performance <cite>(Zhang et al., 2014)</cite> , which costs a lot, and in which it is difficult to get high consistency. In this paper, we carry out an automatic evaluation.