Our study contributes to the increasing literature to show the utility of linguistic resources for deep learning models by providing a deep analysis of a recently proposed model <cite>(Plank and Agić, 2018)</cite> .