Our interest here is the ability of capturing hierarchical structure without being equipped with explicit structural representations <cite>(Bowman et al., 2015b</cite>; Tran et al., 2016; Linzen et al., 2016) .