We study these models' ability of capturing hierarchical structure without explicit parse trees, by evaluating the bi-directional LSTM with attention and the Transformer models on a novel multi-hierarchical reasoning task. To mitigate effects of spurious correlations and annotation artifacts in natural language datasets [1]}, we propose Operating Relational-Cum-Hierarchical Arithmetic Reasoning Dataset (Orchard), a dataset comprising sequences of numbers and nested mathematical operators. The task is as follows: Given two trees with values 0-9 and operators {FIRST, LAST, MIN, MAX, COPY}, evaluate both trees at their roots. An example is illustrated in Figure REF .
<FIGURE>