Different from BeerAdvocate, the annotations of MovieReview are at the phrase-level, which are formed as multiple short spans. In addition, these annotated rationales often tend to be “over-complete”, i.e., they contain multiple phrases, all of which are individually highly predictive of the output. Because of this, the advantage of HardKuma becomes less obvious compared to other baselines. Yet it still outperforms two different implementations of Rnp (i.e., the published result in  [1]}, and our own implementation). Our A2r method consistently beats all the baselines including the strong BERT-based approach.
