We leave the exploration of probing tasks for each ELMo <cite>[33]</cite> layer representation to future research, given that it could provide a framework to expose the linguistic properties capture by each representation level of the LSTM.